Two-dimensional image capture for an augmented reality representation

ABSTRACT

Technologies and implementations for capturing two-dimensional images for an augmented reality representation are generally disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national stage filing under 35 U.S.C. §371 of PCTApplication No. PCT/US2011/39639, filed on Jun. 8, 2011.

BACKGROUND

Unless otherwise indicated herein, the approaches described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Some image processing services may merge several images into a virtualtour. Such virtual tours may attempt to simulate as much of athree-dimensional (3D) an experience as possible. Meanwhile othersystems may convert a series of two-dimensional (2D) images to 3D.

The quality of the 3D model or synthesis produced may depend on thepositioning and resolution of photos. When using photo-to-3D conversion,for example, it is possible to wind up with some combination of missingsurfaces and/or images that lack enough shared context or edges to allowthe integration of a photo into such a 3D model.

As augmented reality representations (e.g., 3D images) may be used formore and more purposes, average home users may want to be able to takephotos of objects and places and generate good quality 3D models ofthem.

SUMMARY

Some example methods, apparatus, and systems described herein may relateto capturing two-dimensional images for an augmented realityrepresentation.

Some example methods, apparatus, and systems related to capturingtwo-dimensional images for an augmented reality representation may beimplemented in an electronic image capturing device. Such methods mayinclude determining, by an electronic image capturing device, anaugmented reality representation of an object based at least in part onan analysis of one or more two-dimensional images of the object. Such anaugmented reality representation may include a plurality of surfacehypotheses. One or more reliability values associated with individualsurface hypotheses may be determined. The one or more reliability valuesmay be compared with one or more threshold reliability criteria toidentify one or more surface areas of interest from the plurality ofsurface hypotheses. Guidance regarding capturing one or more additionaltwo-dimensional images of the object may be displayed via a display ofthe electronic image capturing device. Such guidance may be based atleast in part on the identified surface areas of interest.

Some example methods, apparatus, and systems related to capturingtwo-dimensional images for an augmented reality representation might beimplemented in an electronic image capturing device. Such an electronicimage capturing device may include an optical sensor and a control unit.Such an optical sensor may be configured to capture two-dimensionalimages of an object and capture real-time video data. Such a controlunit may be configured to determine an augmented reality representationof the object based at least in part on an analysis of one or moretwo-dimensional images of the object. Such an augmented realityrepresentation may include a plurality of surface hypotheses. One ormore reliability values associated with individual surface hypothesesmay be determined by the control unit. The one or more reliabilityvalues may be compared with one or more threshold reliability criteriato identify one or more surface areas of interest from the plurality ofsurface hypotheses. Guidance may be determined regarding capturing oneor more additional two-dimensional images of the object via a display ofthe electronic image capturing device. Such guidance may be based atleast in part on the identified surface areas of interest.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the drawings and the followingdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

Subject matter is particularly pointed out and distinctly claimed in theconcluding portion of the specification. The foregoing and otherfeatures of the present disclosure will become more fully apparent fromthe following description and appended claims, taken in conjunction withthe accompanying drawings. Understanding that these drawings depict onlyseveral embodiments in accordance with the disclosure and are,therefore, not to be considered limiting of its scope, the disclosurewill be described with additional specificity and detail through use ofthe accompanying drawings.

In the drawings:

FIG. 1 illustrates an example electronic image capturing device that isarranged in accordance with at least some embodiments of the presentdisclosure;

FIG. 2 illustrates an example electronic image capturing devicecapturing one or more two-dimensional images of an object that isarranged in accordance with at least some embodiments of the presentdisclosure;

FIG. 3 illustrates an example process for capturing two-dimensionalimages for an augmented reality representation that is arranged inaccordance with at least some embodiments of the present disclosure;

FIG. 4 illustrates another example process for capturing two-dimensionalimages for an augmented reality representation that is arranged inaccordance with at least some embodiments of the present disclosure;

FIG. 5 illustrates an example augmented reality representation includinga multiple number of surface hypotheses that is arranged in accordancewith at least some embodiments of the present disclosure;

FIG. 6 illustrates an example viewing cone associated with an augmentedreality representation that is arranged in accordance with at least someembodiments of the present disclosure;

FIG. 7 illustrates example viewing cones associated with surface areasof interest from the multiple number of surface hypotheses that arearranged in accordance with at least some embodiments of the presentdisclosure;

FIG. 8 illustrates an example selection for obtaining data on surfaceareas of interest based on the viewing cones that is arranged inaccordance with at least some embodiments of the present disclosure;

FIG. 9 illustrates an example screen shot that is arranged in accordancewith at least some embodiments of the present disclosure;

FIG. 10 is an illustration of an example computer program product thatis arranged in accordance with at least some embodiments of the presentdisclosure; and

FIG. 11 is a block diagram of an illustrative embodiment of a computingdevice arranged in accordance with at least some embodiments of thepresent disclosure.

DETAILED DESCRIPTION

The following description sets forth various examples along withspecific details to provide a thorough understanding of claimed subjectmatter. It will be understood by those skilled in the art, however, thatclaimed subject matter may be practiced without some or more of thespecific details disclosed herein. Further, in some circumstances,well-known methods, procedures, systems, components and/or circuits havenot been described in detail in order to avoid unnecessarily obscuringclaimed subject matter.

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar components, unless contextdictates otherwise. The illustrative embodiments described in thedetailed description, drawings, and claims are not meant to be limiting.Other embodiments may be utilized, and other changes may be made,without departing from the spirit or scope of the subject matterpresented here. It will be readily understood that the aspects of thepresent disclosure, as generally described herein, and illustrated inthe Figures, can be arranged, substituted, combined, and designed in awide variety of different configurations, all of which are explicitlycontemplated and make part of this disclosure.

This disclosure is drawn, inter alia, to methods, apparatus, and systemsrelated to capturing two-dimensional images for an augmented realityrepresentation.

Methods, apparatus, and systems are described below for computingadvantageous viewpoints for photography in order to build augmentedreality representations (e.g., 3D models) and showing those viewpointsto a user on their electronic image capturing device. For example, asmart phone-type electronic image capturing device may display apreferred vantage point showing a target view that is suggested to bephotographed by the user. Such a preferred vantage point may have beencalculated to provide additional image data for portions of theaugmented reality representation that have low quality data or portionsof the augmented reality representation that were not directly viewedbut have been theorized to exist.

FIG. 1 illustrates an example electronic image capturing device 100 thatis arranged in accordance with at least some embodiments of the presentdisclosure. Electronic image capturing device 100 may be used to performsome or all of the various functions discussed below in connection withFIG. 3 and/or FIG. 4. Electronic image capturing device 100 may includeany device or collection of devices capable of undertaking digital imagecapturing. As used herein the term “electronic image capturing device”may refer to a small-form factor portable electronic device capable ofdigital image taking such as, for example, a digital camera, a cellphone, a personal data assistant (PDA), a mobile computing pad, apersonal media player device, a wireless web-watch device, anapplication specific device, or the like, and/or combinations thereof.

In the illustrated example, electronic image capturing device 100 mayinclude an optical sensor 102, a control unit 104, and/or a display 106.Such an optical sensor 102 may be configured to capture two-dimensionalimages of an object (not illustrated) and/or capture real-time videodata.

Control unit 104 may be configured to determine an augmented realityrepresentation of such an object captured by optical sensor 102. Forexample, control unit 104 may be configured to determine an augmentedreality representation based at least in part on an analysis of one ormore two-dimensional images of the object. As used herein the term“augmented reality representation” may refer to a real-time direct viewor an indirect view of a physical, real-world object, such as athree-dimensional image of an object or the like. Guidance regardingcapturing one or more additional two-dimensional images of the objectmay be provided to a user, such as by visual guidance delivered viadisplay 106. Control unit 104 may utilize such additionaltwo-dimensional images of the object to update the augmented realityrepresentation of such an object.

Further, electronic image capturing device 100 may also includeadditional items such as memory, a processor, network interface logic,etc. that have not been shown in FIG. 1 for the sake of clarity. Forexample, electronic image capturing device 100 may also includeadditional items, such as those described with respect to FIG. 10 below.In some examples, display 106 may be housed within electronic imagecapturing device 100 or display 106 may be an unattached display (e.g.,such as a head-up display or secondary display device).

FIG. 2 illustrates electronic image capturing device 100 capturing oneor more two-dimensional images of an object 200 that is arranged inaccordance with at least some embodiments of the present disclosure. Inthe illustrated example, electronic image capturing device 100 maycapture a two-dimensional image of object 200 taken from a firstposition 202. Electronic image capturing device 100 may be configured todetermine an augmented reality representation based at least in part onan analysis of the two-dimensional image of object 200 taken from firstposition 202.

Guidance regarding capturing one or more additional two-dimensionalimages of the object may be provided to a user 204 via electronic imagecapturing device 100. For example, electronic image capturing device 100may guide user 204 to capture another two-dimensional image of object200 taken from a second position 206. Electronic image capturing device100 may be configured to update the augmented reality representationbased at least in part on an analysis of the two-dimensional image ofobject 200 taken from second position 206.

FIG. 3 illustrates an example process 300 for capturing two-dimensionalimages for an augmented reality representation that is arranged inaccordance with at least some embodiments of the present disclosure. Inthe illustrated example, process 300, and other processes describedherein, set forth various functional blocks or actions that may bedescribed as processing steps, functional operations, events and/oracts, etc., which may be performed by hardware, software, and/orfirmware. Those skilled in the art in light of the present disclosurewill recognize that numerous alternatives to the functional blocks shownin FIG. 3 may be practiced in various implementations. For example,although process 300, as shown in FIG. 3, may comprise one particularorder of blocks or actions, the order in which these blocks or actionsare presented does not necessarily limit claimed subject matter to anyparticular order. Likewise, intervening actions not shown in FIG. 3and/or additional actions not shown in FIG. 3 may be employed and/orsome of the actions shown in FIG. 3 may be eliminated, without departingfrom the scope of claimed subject matter. Process 300 may include one ormore of functional operations as indicated by example operations 302,304, 306, and/or 308.

As illustrated, process 300 may be implemented for capturingtwo-dimensional images for an augmented reality representation that maybe implemented in an electronic image capturing device (see, e.g., FIG.1). In some examples, one or more two-dimensional images of the objectmay be analyzed to form surface hypotheses for an augmented realityrepresentation of an object (e.g., 3D modeling). Such surface hypothesesmay be used to determine surface areas of interest (e.g., missingsurfaces or surfaces with image quality below a threshold reliabilitycriteria). Such surface areas of interest may be used to determineguidance regarding capturing one or more additional two-dimensionalimages, such as viewing locations that will provide needed in-fill offeatures to provide a full model at a target quality, for example. Suchguidance (e.g., viewing locations and/or directions) may be provided toa user so that additional two-dimensional images may be captured and theaugmented reality representation may be updated.

Processing may begin at operation 302, “DETERMINE AN AUGMENTED REALITYREPRESENTATION INCLUDING MULTIPLE SURFACE HYPOTHESES”, where anaugmented reality representation including a multiple number of surfacehypotheses may be determined. For example, such an augmented realityrepresentation of an object may be determined based at least in part onan analysis of one or more two-dimensional images of the object. Such anaugmented reality representation may include multiple surfacehypotheses. In some examples, “pose matching” (or the like) may be usedto determine how two-dimensional images fit into an augmented realityrepresentation (e.g., a 3D scene).

In some examples, the surface hypotheses may be computed employing aRandom Sample Consensus Surface Extraction (RANSAC) algorithm, or thelike, or combinations thereof. For example, RANSAC may take a pointcloud and fit surface hypotheses to the point cloud. The overallapproach can be described as generating a surface that minimizes theleast squares difference from the surface to a point cloud whilesatisfying various assumptions. See, for example, Martin A. Fischler andRobert C. Bolles, “Random sample consensus: A paradigm for model fittingwith applications to image analysis and automated cartography,” Commun.ACM, 24(6): 381-395, 1981. The assumptions can include things like anexpectation of planarity and linear features (common for architecture),and assumptions that recognized symmetry may continue into undetectedareas when not precluded by the data. RANSAC may be amenable to avariety of inputs such as assumptions of symmetry or suggested geometryfrom possible image recognition hits. RANSAC (or other suitable general“error estimation” techniques) may be used in conjunction with lines inan image or edge conditions based on 3D extraction from multipletwo-dimensional images or a video stream. For example, edges can beroughly extracted from video in the electronic image capturing deviceand then RANSAC may use tracked points to finalize shapes, textures,and/or locations of edges and planes.

In some examples, the augmented reality representation may have a groupof surface hypotheses that do not yet have images of their surface orsurfaces with poor image quality. Such surface hypotheses may beunobserved, but some type of geometry may be developed for the augmentedreality representation based at least in part on knowledge of where suchsurfaces can't be (e.g., where such surfaces can't be or it would havebeen seen) and/or assumptions based on other elements of the augmentedreality representation, such as symmetry or surface continuation, forexample.

Processing may continue from operation 302 to operation 304, “DETERMINEONE OR MORE RELIABILITY VALUES ASSOCIATED WITH INDIVIDUAL SURFACEHYPOTHESES”, where one or more reliability values associated withindividual surface hypotheses may be determined. For example, suchreliability values may quantify a quality of surface hypotheses (e.g.,in pixels per area units, or the like). In such an example, suchreliability values may be determined based at least in part on thenumber of pixels available per area units (e.g., in square inches or thelike) associated with an individual surface hypothesis. Such reliabilityvalues may be utilized to determine which surface hypotheses might notmeet quality standards (e.g., because they were imaged at a highincident angle).

Processing may continue from operation 304 to operation 306, “IDENTIFYONE OR MORE SURFACE AREAS OF INTEREST”, where one or more surface areasof interest may be identified. For example, the one or more reliabilityvalues may be compared with one or more threshold reliability criteriato identify one or more surface areas of interest from the surfacehypotheses. For example, such threshold reliability criteria may includeone or more of the following criteria: a pixel per area threshold, anincomplete image data threshold, an absent image data threshold, imagesnot matching at overlaps, image blur level, gamma correction (e.g.,matching luminance or tristimulus values in video or still imagesystems), or the like, and/or combinations thereof. For example, a pixelper area-type reliability value associated with an individual surfacehypothesis may be compared with a pixel per area-type thresholdreliability criteria to determine if that individual surface hypothesisshould be identified as a surface area of interest.

To evaluate quality of the surface hypotheses, an orthogonal transformmay be performed to get an image (which need not be displayed, just incomputation, for example) of what the surface looks like as seen fromthe perpendicular, then an image quality evaluation may be applied tothis image. The quality evaluation may be tested against a metric toclassify each surface as sufficient or not.

In some examples, the surface hypotheses of the augmented realityrepresentation may include higher quality surface hypotheses that wereextracted based on existing image data and lower quality polygon “holes”where there may be no existing image data. Such “holes” (e.g., wherethere is no existing image data) would fail any metric but zeroautomatically, and be identified as surface areas of interest.

Processing may continue from operation 306 to operation 308, “DETERMINEGUIDANCE REGARDING CAPTURING ONE OR MORE ADDITIONAL TWO-DIMENSIONALIMAGES”, where guidance regarding capturing one or more additionaltwo-dimensional images may be determined. For example, guidanceregarding capturing one or more additional two-dimensional images of theobject may be displayed via a display of the electronic image capturingdevice. Such guidance may be based at least in part on the identifiedsurface areas of interest, for example. Process 300 may utilize suchadditional two-dimensional images of the object to update the augmentedreality representation.

As will be discussed in greater detail below with respect to FIG. 9, insome examples, such guidance may indicate one or more vantage points forcapturing the one or more additional two-dimensional images of theobject. Additionally or alternatively, such guidance may indicate thereliability values associated with individual surface hypotheses.

Some additional and/or alternative details related to process 300 may beillustrated in one or more examples of implementations discussed ingreater detail below with regard to FIG. 4.

FIG. 4 illustrates another example process 400 for capturingtwo-dimensional images for an augmented reality representation that isarranged in accordance with at least some embodiments of the presentdisclosure. In the illustrated example, process 400 may include one ormore of operations as illustrated by operations 402, 404, 406, 408, 410,412, 414, and/or 416.

Processing may begin at operation 402, “DETERMINE AN AUGMENTED REALITYREPRESENTATION INCLUDING MULTIPLE SURFACE HYPOTHESES”, where anaugmented reality representation including a multiple number of surfacehypotheses may be determined. For example, such an augmented realityrepresentation of an object may be determined based at least in part onan analysis of one or more two-dimensional images of the object. Such anaugmented reality representation may include multiple surfacehypotheses.

Additionally or alternatively, prior to operation 402, one or moretwo-dimensional images of the object may be selected for determining theaugmented reality representation of the object. For example a user mayindicate an already taken image or may activate a camera mode toestablish that the current viewport is framing something the user wishesto model.

Processing may continue from operation 402 to operation 404, “DETERMINEONE OR MORE RELIABILITY VALUES ASSOCIATED WITH INDIVIDUAL SURFACEHYPOTHESES”, where one or more reliability values associated withindividual surface hypotheses may be determined.

Processing may continue from operation 404 to operation 406, “IDENTIFYONE OR MORE SURFACE AREAS OF INTEREST”, where one or more surface areasof interest may be identified. For example, the one or more reliabilityvalues may be compared with one or more threshold reliability criteriato identify one or more surface areas of interest from the multiplenumber of surface hypotheses.

Processing may continue from operation 406 to operation 408, “DETERMINEONE OR MORE VIEWING CONES”, where one or more viewing cones may bedetermined. For example, one or more viewing cones associated withindividual surface areas of interest may be determined. As used herein,the term “viewing cone” may refer to a defined viewing area bounded by adefined viewing angle and centered by a surface normal vector.

For example, a surface normal vector and viewing angle (e.g., a viewingcone) can be developed from individual surface hypotheses. Such aviewing cone may provide a set of locations and view angles that canguide the user to occupy to observe surface areas of interest. In caseswhere the user reaches this location and view angle and does not observethe surface area of interest, an additional two-dimensional image may becaptured, updating the augmented reality representation and the surfaceareas of interest. Alternatively, in cases where the user reaches thislocation and view angle and does not observe the surface area ofinterest, the user may select another location and view angle associatedwith another surface area of interest.

In some examples, small areas of low quality may generate excessive(e.g., dozens or thousands) surface areas of interest and the associatedviewing cones. In such examples, viewing cones of such excessive surfaceareas of interest may be merged into a minimal intersection to chooseone or a series of fewer additional two-dimensional images that can beused to provide some or all of the missing information. In someexamples, the merging of the viewing cones into groups for establishinga few discrete two-dimensional images to take, may be implemented as aversion of the programming Knapsack problem, solutions to which mayinclude the Greedy algorithm, linear programming (e.g., linearprogramming may be appropriate in some cases due to geometricrestrictions), or the like, and/or combinations thereof.

Additionally or alternatively, a filter may also be applied to require aminimum augmented reality representation impact before justifyingcapturing additional two-dimensional images. Such a filter may preventtiny transition surfaces or surface glare from generating large numbersof surface areas of interest.

Processing may continue from operation 408 to operation 410, “DETERMINEA VIEW RANGE AND ORIENTATION”, where a view range and orientation may bedetermined. For example, a view range and orientation associated withone or more of the viewing cones may be determined based at least inpart on a quality level input.

In some examples, such a quality level input may have preset defaultsettings and/or settings that may be adjusted based on user input. Forexample, a user may indicate an overall image quality desired (e.g.,high-quality, medium-quality, or low-quality, or the like). Additionallyor alternatively, in some examples, it may be possible to obtain thequality level input from another source (e.g., like a game orpublication or program that may take a 3D model as an input) in order toallow such a source to require augmented reality representations of acertain quality. Additionally or alternatively, such quality level inputmay vary depending on the location of the associated individual surfacehypotheses and/or the type of object being represented. For example, theobject may be a vehicle (e.g., which may be independently recognized bythe electronic image capturing device or may be indicated by userinput). The data regarding the underside of such a vehicle-type objectmay not be needed at all, or may have a relatively lower associatedquality level input as compared to other portions of the vehicle-typeobject. Accordingly, such quality level input may vary depending on thelocation of the associated individual surface hypotheses and/or the typeof object being represented. In some examples, such quality level inputmay compared with or adjust the threshold reliability criteria.

Referring back to FIG. 2, for example, a view range 207 betweenelectronic image capturing device 100 and object 200 may be determinedthat is at an optimal distance while also reaching a desired quality.Such a desired quality may be based at least in part on a pixel perarea-based quality level input (e.g., such as pixels spread across area210), for example. Such an optimal distance that also reaches a desiredquality may be based at least in part on a lens angle 208 and resolutionof electronic image capturing device 100. Such lens angle 208 may comefrom lens setting extents and/or a subtended angle of object 200 inframe. For example, view range 207 may be inversely proportional to thequality level input, inversely proportional to lens angle 208, andproportional to resolution of electronic image capturing device 100.

Referring back to FIG. 4, accordingly, such a determination of the viewrange and orientation may be influenced by the resolution of theelectronic image capturing device, lens angle, and/or on the qualitylevel input (e.g., the desired quality). If only a rough outline isdesired, the resultant view range and/or orientation may be different asthe view range and orientation angles may work out much larger, as thepixel density need not be as great (and there may be more overlaps ofview cones—potentially requiring fewer images). For example, such aresultant view range and/or orientation may be based at least in part ona comparison of the quality level input to the size of a pixel atparticular distance and/or lens setting. Accordingly, adjusting the viewrange and orientation associated with one or more of the viewing conesbased at least in part on a quality level input and/or on resolution mayincrease or decrease the number of projected additional two-dimensionalimages (e.g., to achieve the fewest number of projected additionaltwo-dimensional images needed).

Processing may continue from operation 410 to operation 412, “MONITORREAL-TIME VIDEO DATA”, where real-time video data may be monitored. Forexample, real-time video data may be monitored by the electronic imagecapturing device in order to track the relative position and orientationbetween the electronic image capturing device and the object.

Processing may continue from operation 412 to operation 414, “DETERMINEA REAL-TIME RELATIVE POSITION AND ORIENTATION”, where a real-timerelative position and orientation may be determined. For example, areal-time relative position and orientation between the electronic imagecapturing device and the object may be determined based at least in parton the real-time video data. For example, the real-time video data maybe compared with a current augmented reality representation to confirmthat the object is currently being monitored and/or to determine therelative position and orientation between the electronic image capturingdevice and the object. For example, the real-time video data may becompared with a current augmented reality representation via “posematching” (or the like); where such pose matching may be used todetermine how two-dimensional images compare to an augmented realityrepresentation (e.g., a 3D scene). See, for example, Lingyun Liu.,Stamos I., A systematic approach for 2D-image to 3D-range registrationin urban environments. IEEE ICCV, 1-8, 2007.

Processing may continue from operation 414 to operation 416, “DETERMINEGUIDANCE REGARDING CAPTURING ONE OR MORE ADDITIONAL TWO-DIMENSIONALIMAGES”, where guidance regarding capturing one or more additionaltwo-dimensional images may be determined. For example, guidanceregarding capturing one or more additional two-dimensional images of theobject may be displayed via a display of the electronic image capturingdevice. Such guidance may be based at least in part on the identifiedsurface areas of interest, for example. In some examples, the guidancemay be based at least in part on the view range and orientationassociated with one or more of the viewing cones.

In some examples, such guidance may include the user being shown theview cones associated with individual surface areas of interest andasked to fill the view cones in. In some examples, such guidance mayinclude the user being shown a set of locations and view angles that canguide the user to occupy to observe surface areas of interest (e.g.,based at least in part on a corresponding viewing cone). In such anexample, there might be a large number of overlapping possible viewingcones, (e.g., see FIG. 8), and a particular guided location and viewangle may be determined as being most beneficial (e.g., as perhaps thatparticular guided location and view angle is within several view cones).Additionally or alternatively, such guidance may include the user beingshown some sort of overlay indicating the reliability values (or anindication of a comparison of the reliability values of with thethreshold reliability criteria) associated with individual surface areasof interest or the reliability values associated with individual viewcones (e.g., each potential photography location) so that they the usermay decide where to take additional two-dimensional images to fill inthe augmented reality representation.

This guidance may then evolve with each two-dimensional image taken, asthe images associated with particular surface areas of interest orparticular view cones are recognized as having been captured. In such anexample, such guidance may then evolve with each two-dimensional imagetaken, as the augmented reality representation may be updated uponcapturing additional two-dimensional images so that such particularsurface areas of interest or particular view cones may be removed fromindication when a corresponding two-dimensional image is captured.Additionally or alternatively, such guidance may evolve based at leastin part on images captured from real-time video data captured by theelectronic image capturing device.

In some examples, such guidance may only display a top one, two, orthree view cones, which might be updated upon capturing an image fromone of such view cones. Additionally or alternatively, such guidance maythen evolve with each two-dimensional image taken, as the augmentedreality representation may be updated upon capturing additionaltwo-dimensional images, which may result in a new set of surface areasof interest and/or view cones.

As will be discussed in greater detail below with respect to FIG. 9, insome examples, the guidance may be based at least in part on the viewrange and orientation associated with one or more of the viewing cones,as discussed above with respect to operations 408 and 410. Additionallyor alternatively, the guidance may be based at least in part on thedetermined real-time relative position and orientation between theelectronic image capturing device and the object, as discussed abovewith respect to operations 412 and 414.

Referring back to FIG. 1, as described with reference to processes 300and/or 400 of FIGS. 3 and 4, an augmented reality representation mayinclude a multiple number of surface hypotheses. One or more reliabilityvalues may be associated with individual surface hypotheses, which maybe determined by control unit 104. The one or more reliability valuesmay be compared with one or more threshold reliability criteria toidentify one or more surface areas of interest from the multiple numberof surface hypotheses. Guidance regarding capturing one or moreadditional two-dimensional images of the object may be provided to auser, such as by visual guidance delivered via display 106. Suchguidance may be determined based at least in part on the identifiedsurface areas of interest. For example, such guidance may prompt a userto capture one or more additional two-dimensional images of the objectso as to obtain additional data regarding the identified surface areasof interest.

FIG. 5 illustrates an example augmented reality representation 500including a multiple number of surface hypotheses that is arranged inaccordance with at least some embodiments of the present disclosure. Inthe illustrated example, augmented reality representation 500 isillustrated by a first view 502, a second view 504, and a third view506.

A number of different approaches may be employed to determine desiredlocations to capture one or more additional two-dimensional images inorder to cover surface areas of interest (e.g., unobserved surfaces orlow quality surfaces). This may be done based on information fromobserved surfaces. For example, RANSAC is the abbreviation for “RandomSample Consensus”, which may be used for taking a point cloud andfitting surface hypotheses to the point cloud. The overall approach maybe described as generating a surface that minimizes the least squaresdifference from the surface to a point cloud while satisfying variousassumptions. The assumptions may include an expectation of planarity andlinear features (common for architecture), recognition of symmetry (asin FIG. 5), and comparable ones, which are likely to continue intounobserved areas when not precluded by the data. In order to determinedesired locations to capture one or more additional two-dimensionalimages, surfaces that are in some way deficient may be identified suchthat more data on those surfaces can be collected.

The three dimensional figures in augmented reality representation 500illustrate varying reliability values associated with individual surfacehypothesis by varying the presentation of the individual surfacehypothesis. According to an example scenario, such reliability valuesmay be computed as the sum of the area proportion with point densitybelow average and the ratio of the surface mean error to the surfacemean error on surfaces closest to normal with the measured view. Thus,both the quality of the points and the proportion of the surface thathas not been observed may be used in determining such reliabilityvalues.

For example, an augmented reality representation 500 generated based ona two-dimensional images similar to first view 502 may have a relativelyhigh reliability value associated with surface 510, a relatively lowreliability value associated with surface 512 and 516, and a relativelyintermediate reliability value associated with surface 514. For example,surface 510 is easily seen from first view 502, while surface 512 canonly be partially viewed or viewed from an extreme angle and surface 516can only be predicted as it is not directly viewable, limiting thequality of data associated with surface 512 and surface 516.

Views 502, 504, and 506 of augmented reality representation 500 may beobtained as an output of a RANSAC surface extractor program. In views502, 504, and 506, surfaces of the same architectural structure arehypothesized (referred to herein as surface hypothesis) and thestructure is shown in various rotated positions to illustrate surfaceareas of interest (e.g., hypothesized but unobserved surfaces wheresurface image data has not been taken or low quality surfaces).

In addition to computing a suitable geometry, a reliability value may begenerated for individual surface hypothesis segments (shown in differentshades of grey). Instead of only using a least squares matching forconfidence (which can actually be quite high for a presumed surface withfew points), an appropriate reliability value (e.g., based on imagepixels per area) may be used. In such an example, both the quality ofthe points (which is low here for the surface 512, even though it hasbeen directly observed) and the proportion of the surface that has notbeen observed (which is high for surface 514 and surface 516) may beevaluated.

In the illustrated example, reliability values may be illustrated byvarying shades of grey for individual surface hypothesis segments. Arelatively high reliability value associated with surface 510 may beillustrated with a light shade of grey, a relatively low reliabilityvalue associated with surface 512 and 516 might be illustrated with adark shades of grey, and a relatively intermediate reliability valueassociated with surface 514 might be illustrated with an intermediateshade of grey. In some embodiments, such reliability values may beillustrated by varying shades of color (e.g., red, orange, yellow, andgreen). Additionally or alternatively, such reliability values may beillustrated by numerical or percentage values representing thereliability value appearing on or near the associated surface hypothesissegments or associated with an indicated cone or photo viewpoint.

FIG. 6 illustrates an example viewing cone associated with an augmentedreality representation that is arranged in accordance with at least someembodiments of the present disclosure. In the illustrated example,augmented reality representation 600 is illustrated by a first view 602and a second view 606. View cone 604 associated with the low qualitysurface 512, is shown in a broad view associated with first view 602 andis shown in an end view (e.g., a view centered by a surface normalvector with respect to surface 512) associated with second view 606. Asingle view cone 604 is illustrated here for clarity. In this case, viewcone 604 may be a draft extension grown from a surface normal vector 610from the existing surface 512 with a defined view angle 612. Such adefined view angle 612 may be calculated based at least in part onresolution and/or distance so as to yield a desired surface quality orresolution. Views, which might capture better data for surface areas ofinterest (e.g., surface 512) in order to complete an augmented realityrepresentation for the object, may be determined based at least in parton defined viewing angle 612 and surface normal vector 610 computed fromsuch surface areas of interest.

FIG. 7 illustrates example viewing cones associated with surface areasof interest from the multiple number of surface hypotheses that arearranged in accordance with at least some embodiments of the presentdisclosure. In the illustrated example, augmented reality representation700 is illustrated by a first view 702, a second view 704, and a thirdview 706. A multiple number of view cones 708 may be associated withsurface areas of interest (e.g., unobserved and low quality surfaces(512, 514, 516—of FIG. 5, for example).

Individual views cones 708 may be drawn using the surface normal vectorassociated with each surface area of interest (e.g., unobserved and lowquality surfaces (512, 514, 516—of FIG. 5, for example). For non-planarsurfaces this may be done by breaking down the surfaces that werepreviously identified as low quality from a single score (above) into aVoronoi tessellation (or the like) and giving each region a similarscore and then using the largest area in the Voronoi diagram with ascore below the threshold as the surface for normal generation, forexample (e.g., generation of the cone from the left half of surface 514in FIG. 5). The segments of the Voronoi diagram may be the points in theplane that are equidistant to the two nearest sites. Then, each surfacenormal vector may be extended and the corresponding view cone 702defined based on the imaging metrics discussed above.

FIG. 8 illustrates an example selection for obtaining data on surfaceareas of interest based on the viewing cones that is arranged inaccordance with at least some embodiments of the present disclosure. Inthe illustrated example, augmented reality representation 800 isillustrated by a first view 802. A multiple number of view cones 708 maybe associated with surface areas of interest (e.g., unobserved and lowquality surfaces (512, 514, 516—of FIG. 5, for example).

Some or all of the multiple number of view cones 708 may overlap at anintersection 804. The selection of desired locations for capturingimages may be based at least in part on finding locations that generateas much data on as many surface areas of interest as possible (e.g.,such as via an intersection 804). In algorithmic terms, if filling inone of view cones 708 is allotted as having a certain “score” then eachavailable view cone 708 may have an overall compound score that mayrepresent how much data that view cone 708 contributes to the augmentedreality representation. The possible locations may also have otherlimitation on them such as height of the photographer and exclusion oftaking two-dimensional images from inside other objects. Taking thehighest scoring locations into mind may result in filling in theaugmented reality representation with the least images, such as wheresome or all of the multiple number of view cones 708 may overlap atintersection 804, for example.

FIG. 9 illustrates an example screen shot that is arranged in accordancewith at least some embodiments of the present disclosure. As discussedabove, in some examples, such guidance may indicate one or more vantagepoints for capturing the one or more additional two-dimensional imagesof the object. Additionally or alternatively, such guidance may indicatethe reliability values associated with individual surface hypotheses.

As discussed above (with respect to FIG. 5), a reliability value may begenerated for individual surface hypothesis segments (shown in differentshades of grey). In the illustrated example, reliability values may beillustrated by varying shades of grey for individual surface hypothesissegments. A relatively low reliability value associated with surface 512and 516 might be illustrated with a dark shades of grey, and arelatively intermediate reliability value associated with surface 514might be illustrated with an intermediate shade of grey. In someembodiments, such reliability values may be illustrated by varyingshades of color (e.g., red, orange, yellow, and green). In such anexample, the color red might be associated with an unobserved surface516, the color orange might illustrate a relatively low reliabilityvalue associated with surface 512, the color yellow might illustrate arelatively intermediate reliability value associated with surface 514,and the color green might illustrate a relatively high reliabilityvalue. Additionally or alternatively, such reliability values may beillustrated by numerical or percentage values representing thereliability value appearing on or near the associated surface hypothesissegments.

In some examples, such guidance may include a display of a currentaugmented reality representation 902. Such a current augmented realityrepresentation 902 may include illustrated reliability values associatedwith surfaces 510, 512, and/or 514, for example. Current augmentedreality representation 902 may be rotated by user interaction with theelectronic image capturing device (e.g., via touch screen or the like),so that a user might see visually from which angles individual surfaceareas of interest (e.g., surfaces 512 and/or 514) might best be viewed.

Additionally or alternatively, in some examples, such guidance mayinclude a display of current augmented reality representation 904 thatmay be oriented to represent a suggested vantage point to capture one ormore additional two-dimensional images. In the illustrated example,current augmented reality representation 904 may be oriented so thatindividual surface areas of interest (e.g., surfaces 512, 514, and/or516) might best be viewed.

Additionally or alternatively, in some examples, such guidance mayinclude a display of object 906 (e.g., a live display). View cone 604may be superimposed on object 906 to represent a suggested vantage pointto capture one or more additional two-dimensional images. Additionallyor alternatively, other visual guidance may include the user being showna location and view angle that can guide the user to occupy to observesurface areas of interest (e.g., based at least in part on acorresponding viewing cone). In such examples, the guidance may be basedat least in part on the determined real-time relative position andorientation between the electronic image capturing device and theobject. For example, the electronic image capturing device may instructa user to keep the electronic image capturing device focused on theobject as the user changes location, so that the display of object 906and view cone 604 may be dynamically altered as relative position andorientation between the electronic image capturing device and the objectchanges.

Additionally or alternatively, view cone 604 may be superimposed oncurrent augmented reality representation 902 and/or 904 to represent asuggested vantage point to capture one or more additionaltwo-dimensional images. In some examples, a multiple number of viewcones 604 may be presented. In such an example, a visual indication maybe given when an image is captured that satisfies each of the multiplenumber of view cones 604.

In some examples, the user can be guided to satisfy each two-dimensionalimage by showing arrows 912 indicating to shift or rotate the electronicimage capturing device, hold it higher or lower, or to approach or moveaway from the object. The desired distance from an object may be a tradeoff between a wider view and higher resolution, and distance may becalculated by comparing the required metric (e.g., in pixels per lineardistance) to the electronic image capturing device resolution and theangle of view to surfaces in the frame. Such a resolution metric may seta maximum distance while minimum distance influences range of view andthus how many images may need to be acquired. In such an example, theguidance may be based at least in part on the determined real-timerelative position and orientation between the electronic image capturingdevice and the object. For example, the electronic image capturingdevice may instruct a user to keep the electronic image capturing devicefocused on the object as the user changes location, so that the displayof arrows 912 (indicating to shift or rotate the electronic imagecapturing device, hold it higher or lower, or to approach or move awayfrom the object) may be dynamically altered as relative position andorientation between the electronic image capturing device and the objectchanges.

In operation, display 106 may be utilized to provide a dynamic feedbackthat may give the user better results and less frustration. Additionallyor alternatively, real-time updating of the augmented realityrepresentations (e.g., 3D models) within the electronic image capturingdevice 100 (FIG. 1), may allow augmented reality representations to beviewed or shared immediately after generation, and/or permit aniterative approach to improving an augmented reality representation aseach additional two-dimensional image is captured.

In operation, providing real-time user feedback to help a user generatean augmented reality representation using two-dimensional images mayreduce user result frustration and a may allow for a higher level ofquality from “crowd-sourced” 3D objects. Such augmented realityrepresentations may be used with electronic games, where users mightcreate augmented reality representations of gear, places, or people anduse them in a game; general photography, where users might generate 3Dlocation and feature imagery for sharing with friends; and/ordocumentation, where users may record positions after a car accident orto record a location for annotation or later comparison to “before.”

FIG. 10 illustrates an example computer program product 1000 that isarranged in accordance with at least some examples of the presentdisclosure. Program product 1000 may include a signal bearing medium1002. Signal bearing medium 1002 may include one or moremachine-readable instructions 1004, which, if executed by one or moreprocessors, may operatively enable a computing device to provide thefunctionality described above with respect to FIG. 3 and/or FIG. 4.Thus, for example, referring to the system of FIG. 1, electronic imagecapturing device 100 may undertake one or more of the actions shown inFIG. 3 and/or FIG. 4 in response to instructions 1004 conveyed by medium1002.

In some implementations, signal bearing medium 1002 may encompass anon-transitory computer-readable medium 1006, such as, but not limitedto, a hard disk drive, a Compact Disc (CD), a Digital Versatile Disk(DVD), a digital tape, memory, etc. In some implementations, signalbearing medium 1002 may encompass a recordable medium 1008, such as, butnot limited to, memory, read/write (R/W) CDs, R/W DVDs, etc. In someimplementations, signal bearing medium 1002 may encompass communicationsmedium 1010, such as, but not limited to, a digital and/or an analogcommunication medium (e.g., a fiber optic cable, a waveguide, a wiredcommunications link, a wireless communication link, etc.).

FIG. 11 is a block diagram illustrating an example computing device1100, such as might be embodied by a person skilled in the art, which isarranged in accordance with at least some embodiments of the presentdisclosure. In one example configuration 1101, computing device 1100 mayinclude one or more processors 1110 and system memory 1120. A memory bus1130 may be used for communicating between the processor 1110 and thesystem memory 1120.

Depending on the desired configuration, processor 1110 may be of anytype including but not limited to a microprocessor (μP), amicrocontroller (μC), a digital signal processor (DSP), or anycombination thereof. Processor 1110 may include one or more levels ofcaching, such as a level one cache 1111 and a level two cache 1112, aprocessor core 1113, and registers 1114. The processor core 1113 mayinclude an arithmetic logic unit (ALU), a floating point unit (FPU), adigital signal processing core (DSP Core), or any combination thereof. Amemory controller 1115 may also be used with the processor 1110, or insome implementations the memory controller 1115 may be an internal partof the processor 1110.

Depending on the desired configuration, the system memory 1120 may be ofany type including but not limited to volatile memory (such as RAM),non-volatile memory (such as ROM, flash memory, etc.) or any combinationthereof. System memory 1120 may include an operating system 1121, one ormore applications 1122, and program data 1124. Application 1122 mayinclude an image guidance algorithm 1123 that is arranged to perform thefunctions as described herein including the functional blocks and/oractions described with respect to process 300 of FIG. 3 and/or process400 of FIG. 4. Program Data 1124 may include image data 1125 for usewith image guidance algorithm 1123. In some example embodiments,application 1122 may be arranged to operate with program data 1124 on anoperating system 1121 such that implementations of providing guidance tocapture two-dimensional images for an augmented reality representationmay be provided as described herein. For example, electronic imagecapturing device 100 (see, e.g., FIG. 1) may comprise all or a portionof computing device 1100 and be capable of performing all or a portionof application 1122 such that implementations of providing guidance tocapture two-dimensional images for an augmented reality representationmay be provided as described herein. This described basic configurationis illustrated in FIG. 11 by those components within dashed line 1101.

Computing device 1100 may have additional features or functionality, andadditional interfaces to facilitate communications between the basicconfiguration 1101 and any required devices and interfaces. For example,a bus/interface controller 1140 may be used to facilitate communicationsbetween the basic configuration 1101 and one or more data storagedevices 1150 via a storage interface bus 1141. The data storage devices1150 may be removable storage devices 1151, non-removable storagedevices 1152, or a combination thereof. Examples of removable storageand non-removable storage devices include magnetic disk devices such asflexible disk drives and hard-disk drives (HDD), optical disk drivessuch as compact disk (CD) drives or digital versatile disk (DVD) drives,solid state drives (SSD), and tape drives to name a few. Examplecomputer storage media may include volatile and nonvolatile, removableand non-removable media implemented in any method or technology forstorage of information, such as computer readable instructions, datastructures, program modules, or other data.

System memory 1120, removable storage 1151 and non-removable storage1152 are all examples of computer storage media. Computer storage mediaincludes, but is not limited to, RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which maybe used to store the desired information and which may be accessed bycomputing device 1100. Any such computer storage media may be part ofdevice 1100.

Computing device 1100 may also include an interface bus 1142 forfacilitating communication from various interface devices (e.g., outputinterfaces, peripheral interfaces, and communication interfaces) to thebasic configuration 1101 via the bus/interface controller 1140. Exampleoutput interfaces 1160 may include a graphics processing unit 1161 andan audio processing unit 1162, which may be configured to communicate tovarious external devices such as a display or speakers via one or moreNV ports 1163. Example peripheral interfaces 1170 may include a serialinterface controller 1171 or a parallel interface controller 1172, whichmay be configured to communicate with external devices such as inputdevices (e.g., keyboard, mouse, pen, voice input device, touch inputdevice, etc.) or other peripheral devices (e.g., printer, scanner, etc.)via one or more I/O ports 1173. An example communication interface 1180includes a network controller 1181, which may be arranged to facilitatecommunications with one or more other computing devices 1190 over anetwork communication via one or more communication ports 1182. Acommunication connection is one example of a communication media.Communication media may typically be embodied by computer readableinstructions, data structures, program modules, or other data in amodulated data signal, such as a carrier wave or other transportmechanism, and may include any information delivery media. A “modulateddata signal” may be a signal that has one or more of its characteristicsset or changed in such a manner as to encode information in the signal.By way of example, and not limitation, communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), infrared (IR) andother wireless media. The term computer readable media as used hereinmay include both storage media and communication media.

Computing device 1100 may be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that includes any of the abovefunctions. Computing device 1100 may also be implemented as a personalcomputer including both laptop computer and non-laptop computerconfigurations. In addition, computing device 1100 may be implemented aspart of a wireless base station or other wireless system or device.

Some portions of the foregoing detailed description are presented interms of algorithms or symbolic representations of operations on databits or binary digital signals stored within a computing system memory,such as a computer memory. These algorithmic descriptions orrepresentations are examples of techniques used by those of ordinaryskill in the data processing arts to convey the substance of their workto others skilled in the art. An algorithm is here, and generally, isconsidered to be a self-consistent sequence of operations or similarprocessing leading to a desired result. In this context, operations orprocessing involve physical manipulation of physical quantities.Typically, although not necessarily, such quantities may take the formof electrical or magnetic signals capable of being stored, transferred,combined, compared or otherwise manipulated. It has proven convenient attimes, principally for reasons of common usage, to refer to such signalsas bits, data, values, elements, symbols, characters, terms, numbers,numerals or the like. It should be understood, however, that all ofthese and similar terms are to be associated with appropriate physicalquantities and are merely convenient labels. Unless specifically statedotherwise, as apparent from the following discussion, it is appreciatedthat throughout this specification discussions utilizing terms such as“processing,” “computing,” “calculating,” “determining” or the likerefer to actions or processes of a computing device, that manipulates ortransforms data represented as physical electronic or magneticquantities within memories, registers, or other information storagedevices, transmission devices, or display devices of the computingdevice.

Claimed subject matter is not limited in scope to the particularimplementations described herein. For example, some implementations maybe in hardware, such as employed to operate on a device or combinationof devices, for example, whereas other implementations may be insoftware and/or firmware. Likewise, although claimed subject matter isnot limited in scope in this respect, some implementations may includeone or more articles, such as a signal bearing medium, a storage mediumand/or storage media. This storage media, such as CD-ROMs, computerdisks, flash memory, or the like, for example, may have instructionsstored thereon, that, when executed by a computing device, such as acomputing system, computing platform, or other system, for example, mayresult in execution of a processor in accordance with claimed subjectmatter, such as one of the implementations previously described, forexample. As one possibility, a computing device may include one or moreprocessing units or processors, one or more input/output devices, suchas a display, a keyboard and/or a mouse, and one or more memories, suchas static random access memory, dynamic random access memory, flashmemory, and/or a hard drive.

There is little distinction left between hardware and softwareimplementations of aspects of systems; the use of hardware or softwareis generally (but not always, in that in certain contexts the choicebetween hardware and software can become significant) a design choicerepresenting cost vs. efficiency tradeoffs. There are various vehiclesby which processes and/or systems and/or other technologies describedherein can be effected (e.g., hardware, software, and/or firmware), andthat the preferred vehicle will vary with the context in which theprocesses and/or systems and/or other technologies are deployed. Forexample, if an implementer determines that speed and accuracy areparamount, the implementer may opt for a mainly hardware and/or firmwarevehicle; if flexibility is paramount, the implementer may opt for amainly software implementation; or, yet again alternatively, theimplementer may opt for some combination of hardware, software, and/orfirmware.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowcharts,and/or examples. Insofar as such block diagrams, flowcharts, and/orexamples contain one or more functions and/or operations, it will beunderstood by those within the art that each function and/or operationwithin such block diagrams, flowcharts, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment,several portions of the subject matter described herein may beimplemented via Application Specific Integrated Circuits (ASICs), FieldProgrammable Gate Arrays (FPGAs), digital signal processors (DSPs), orother integrated formats. However, those skilled in the art willrecognize that some aspects of the embodiments disclosed herein, inwhole or in part, can be equivalently implemented in integratedcircuits, as one or more computer programs running on one or morecomputers (e.g., as one or more programs running on one or more computersystems), as one or more programs running on one or more processors(e.g., as one or more programs running on one or more microprocessors),as firmware, or as virtually any combination thereof, and that designingthe circuitry and/or writing the code for the software and or firmwarewould be well within the skill of one of skill in the art in light ofthis disclosure. In addition, those skilled in the art will appreciatethat the mechanisms of the subject matter described herein are capableof being distributed as a program product in a variety of forms, andthat an illustrative embodiment of the subject matter described hereinapplies regardless of the particular type of signal bearing medium usedto actually carry out the distribution. Examples of a signal bearingmedium include, but are not limited to, the following: a recordable typemedium such as a flexible disk, a hard disk drive (HDD), a Compact Disc(CD), a Digital Versatile Disk (DVD), a digital tape, a computer memory,etc.; and a transmission type medium such as a digital and/or an analogcommunication medium (e.g., a fiber optic cable, a waveguide, a wiredcommunications link, a wireless communication link, etc.).

Those skilled in the art will recognize that it is common within the artto describe devices and/or processes in the fashion set forth herein,and thereafter use engineering practices to integrate such describeddevices and/or processes into data processing systems. That is, at leasta portion of the devices and/or processes described herein can beintegrated into a data processing system via a reasonable amount ofexperimentation. Those having skill in the art will recognize that atypical data processing system generally includes one or more of asystem unit housing, a video display device, a memory such as volatileand non-volatile memory, processors such as microprocessors and digitalsignal processors, computational entities such as operating systems,drivers, graphical user interfaces, and applications programs, one ormore interaction devices, such as a touch pad or screen, and/or controlsystems including feedback loops and control motors (e.g., feedback forsensing position and/or velocity; control motors for moving and/oradjusting components and/or quantities). A typical data processingsystem may be implemented utilizing any suitable commercially availablecomponents, such as those typically found in datacomputing/communication and/or network computing/communication systems.

The herein described subject matter sometimes illustrates differentcomponents contained within, or connected with, different othercomponents. It is to be understood that such depicted architectures aremerely exemplary, and that in fact many other architectures can beimplemented which achieve the same functionality. In a conceptual sense,any arrangement of components to achieve the same functionality iseffectively “associated” such that the desired functionality isachieved. Hence, any two components herein combined to achieve aparticular functionality can be seen as “associated with” each othersuch that the desired functionality is achieved, irrespective ofarchitectures or intermedial components. Likewise, any two components soassociated can also be viewed as being “operably connected”, or“operably coupled”, to each other to achieve the desired functionality,and any two components capable of being so associated can also be viewedas being “operably couplable”, to each other to achieve the desiredfunctionality. Specific examples of operably couplable include but arenot limited to physically mateable and/or physically interactingcomponents and/or wirelessly interactable and/or wirelessly interactingcomponents and/or logically interacting and/or logically interactablecomponents.

With respect to the use of substantially any plural and/or singularterms herein, those having skill in the art can translate from theplural to the singular and/or from the singular to the plural as isappropriate to the context and/or application. The varioussingular/plural permutations may be expressly set forth herein for sakeof clarity.

It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.). It will be further understood by those within the art that if aspecific number of an introduced claim recitation is intended, such anintent will be explicitly recited in the claim, and in the absence ofsuch recitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to inventions containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should typically be interpreted to mean “atleast one” or “one or more”); the same holds true for the use ofdefinite articles used to introduce claim recitations. In addition, evenif a specific number of an introduced claim recitation is explicitlyrecited, those skilled in the art will recognize that such recitationshould typically be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, typically means at least two recitations, or two or morerecitations). Furthermore, in those instances where a conventionanalogous to “at least one of A, B, and C, etc.” is used, in generalsuch a construction is intended in the sense one having skill in the artwould understand the convention (e.g., “a system having at least one ofA, B, and C” would include but not be limited to systems that have Aalone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.). In those instances where aconvention analogous to “at least one of A, B, or C, etc.” is used, ingeneral such a construction is intended in the sense one having skill inthe art would understand the convention (e.g., “a system having at leastone of A, B, or C” would include but not be limited to systems that haveA alone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.). It will be furtherunderstood by those within the art that virtually any disjunctive wordand/or phrase presenting two or more alternative terms, whether in thedescription, claims, or drawings, should be understood to contemplatethe possibilities of including one of the terms, either of the terms, orboth terms. For example, the phrase “A or B” will be understood toinclude the possibilities of “A” or “B” or “A and B.”

Reference in the specification to “an implementation,” “oneimplementation,” “some implementations,” or “other implementations” maymean that a particular feature, structure, or characteristic describedin connection with one or more implementations may be included in atleast some implementations, but not necessarily in all implementations.The various appearances of “an implementation,” “one implementation,” or“some implementations” in the preceding description are not necessarilyall referring to the same implementations.

While certain exemplary techniques have been described and shown hereinusing various methods and systems, it should be understood by thoseskilled in the art that various other modifications may be made, andequivalents may be substituted, without departing from claimed subjectmatter. Additionally, many modifications may be made to adapt aparticular situation to the teachings of claimed subject matter withoutdeparting from the central concept described herein. Therefore, it isintended that claimed subject matter not be limited to the particularexamples disclosed, but that such claimed subject matter also mayinclude all implementations falling within the scope of the appendedclaims, and equivalents thereof.

What is claimed:
 1. A method, comprising: determining, by an electronicimage capturing device, an augmented reality representation of an objectbased at least in part on an analysis of one or more two-dimensionalimages of the object, wherein the augmented reality representationincludes a plurality of surface hypotheses; determining one or morereliability values associated with individual surface hypotheses;comparing the one or more reliability values with one or more thresholdreliability criteria to identify one or more surface areas of interestfrom the plurality of surface hypotheses; and displaying guidanceregarding capturing one or more additional two-dimensional images of theobject via a display of the electronic image capturing device, whereinthe guidance is based at least in part on the identified surface areasof interest; wherein the guidance indicates the one or more reliabilityvalues associated with individual surface hypotheses of the one or moresurface hypotheses and wherein the one or more reliability values arecalculated for the individual surface hypotheses based on an areaproportion comprising a point density below an average point density anda surface mean error.
 2. The method of claim 1, further comprising:determining one or more viewing cones associated with individual surfaceareas of interest; and determining a view range and orientationassociated with one or more of the viewing cones based at least in parton a quality level input, wherein the guidance is based at least in parton the view range and orientation associated with one or more of theviewing cones.
 3. The method of claim 2, wherein the one or more viewingcones individually comprise a defined viewing area bounded by a definedviewing angle and centered by a surface normal vector.
 4. The method ofclaim 1, further comprising: monitoring real-time video data; anddetermining a real-time relative position and orientation between theelectronic image capturing device and the object based at least in parton the real-time video data, wherein the guidance is based at least inpart on the determined real-time relative position and orientationbetween the electronic image capturing device and the object.
 5. Themethod of claim 1, wherein the augmented reality representationcomprises a three-dimensional image of the object.
 6. The method ofclaim 1, wherein the plurality of surface hypotheses are computedemploying a Random Sample Consensus Surface Extraction (RANSAC)algorithm.
 7. The method of claim 1, wherein the one or more thresholdreliability criteria comprise one or more of the following criteria: apixel per area threshold, an incomplete image data threshold, and/or anabsent image data threshold.
 8. The method of claim 1, wherein theguidance indicates one or more vantage points for capturing the one ormore additional two-dimensional images of the object.
 9. The method ofclaim 1, further comprising receiving an indication from a user that oneor more two-dimensional images of the object have been selected, whereinthe determining, by an electronic image capturing device, of theaugmented reality representation of the object is based at least in parton the indication from the user.
 10. An electronic image capturingdevice, comprising: an optical sensor configured to capturetwo-dimensional images of an object and capture real-time video data;and a control unit configured to: determine an augmented realityrepresentation of the object based at least in part on an analysis ofone or more two-dimensional images of the object, wherein the augmentedreality representation includes a plurality of surface hypotheses;determine one or more reliability values associated with individualsurface hypotheses; compare of the one or more reliability values withone or more threshold reliability criteria to identify one or moresurface areas of interest from the plurality of surface hypotheses; anddetermine guidance regarding capturing one or more additionaltwo-dimensional images of the object via a display of the electronicimage capturing device, wherein the guidance is based at least in parton the identified surface areas of interest; wherein the guidanceindicates the one or more reliability values associated with individualsurface hypotheses of the one or more surface hypotheses and wherein theone or more reliability values are calculated for the individual surfacehypotheses based on an area proportion comprising a point density belowan average point density and a surface mean error.
 11. The electronicimage capturing device of claim 10, wherein the control unit is furtherconfigured to: determine one or more viewing cones associated withindividual surface areas of interest; and determine a view range andorientation associated with one or more of the viewing cones based atleast in part on a quality level input, wherein the guidance is based atleast in part on the view range and orientation associated with one ormore of the viewing cones.
 12. The electronic image capturing device ofclaim 11, wherein the one or more viewing cones individually comprise adefined viewing area bounded by a defined viewing angle and centered bya surface normal vector.
 13. The electronic image capturing device ofclaim 10, wherein the control unit is further configured to: monitorreal-time video data; and determine a real-time relative position andorientation between the electronic image capturing device and the objectbased at least in part on the real-time video data, wherein the guidanceis based at least in part on the determined real-time relative positionand orientation between the electronic image capturing device and theobject.
 14. The electronic image capturing device of claim 10, whereinthe augmented reality representation comprises a three-dimensional imageof the object, wherein the guidance regarding capturing the one or moreadditional two-dimensional images is configured to direct a user tocapture the one or more two-dimensional images from a particularperspective wherein the one or more two-dimensional images areidentified to supplement data associated with the plurality of surfacehypotheses to increase the one or more reliability values.
 15. Theelectronic image capturing device of claim 10, wherein the plurality ofsurface hypotheses are computed employing a Random Sample ConsensusSurface Extraction (RANSAC) algorithm.
 16. The electronic imagecapturing device of claim 10, wherein the one or more thresholdreliability criteria comprise one or more of the following criteria: apixel per area threshold, an incomplete image data threshold, and/or anabsent image data threshold.
 17. The electronic image capturing deviceof claim 10, wherein the guidance indicates one or more vantage pointsfor capturing the one or more additional two-dimensional images of theobject.
 18. The electronic image capturing device of claim 10, whereinthe guidance indicates the reliability values associated with individualsurface hypotheses.
 19. The electronic image capturing device of claim10, wherein the electronic image capturing device comprises a digitalcamera, a cell phone, a personal data assistant, a mobile computing pad,a personal media player device, or a wireless web-watch device.
 20. Anarticle comprising: a non-transitory computer readable medium comprisingmachine-readable instructions stored thereon, which, if executed by oneor more processors, operatively enable a computing device to: determinean augmented reality representation of an object based at least in parton an analysis of one or more two-dimensional images of the object,wherein the augmented reality representation includes a plurality ofsurface hypotheses; determine one or more reliability values associatedwith individual surface hypotheses wherein the reliability values arebased at least in part on the number of pixels available per area units;compare of the one or more reliability values with one or more thresholdreliability criteria to identify one or more surface areas of interestfrom the plurality of surface hypotheses; and determine guidanceregarding capturing one or more additional two-dimensional images of theobject via a display of the electronic image capturing device, whereinthe guidance is based at least in part on the identified surface areasof interest, wherein the additional two-dimensional image is identifiedto supplement data associated with the plurality of surface hypothesesto increase the one or more reliability values.